Tivoli Endpoint Manager (TEM) is designed to be a real time (seconds and minutes) system capable of workng in extremely limited bandwidth environments. Properly configured, The TEM platform can effectively and quickly deploy files that are hundreds of MBs large to tens of thousands of computers without significantly affecting general network performance. Because of the speed and prevalence of the TEM platform in the network, however, it is very important that network engineers understand the different types of traffic that the TEM Servers, Relays, Clients, and Consoles can generate.

The following information describes the different types of network traffic and explains the purpose of the traffic as well as information about how to handle "Possible Worst Case Traffic".

Note: The "possible worst case traffic" are definitely not expected; however, in the interest of preparing for the worst case scenario, example situations are given as well mechanisms to recover from the situation. This information is intended to educate network engineers and provide the basis of a disaster recovery procedure related to the Tivoli Endpoint Manager infrastructure.

Please refer to the following diagram in the network discussions below:

 

A

Network Traffic: Gather / Post / Download
Purpose: The TEM Client will contact its TEM Relay (or TEM Server) for several reasons including: gathering the latest information about actions and Fixlet messages, sending results about the computer properties or status of actions/Fixlets, and downloading files (such as patches).
Port: 52311 (configurable by the TEM Administrator at install time)
Protocol: HTTP (TCP/IP)
Origination: The TEM Client initiates the request to the TEM Relay (or TEM Server).
Expected Traffic Frequency:





  • Gather: The TEM Clients will gather the latest Fixlet content or actions once a day OR whenever a new action or new Fixlets are available from the TEM Relay. Each gather is approximately 1 KB - 3 KB (only compressed differences are gathered). An active deployment will have this occur many times (up to many hundreds of times a day when the TEM Console operators are very active)
  • Post: The TEM Clients will send results of their actions / Fixlets (approximately 1 KB each submission) for each action. The TEM Clients will also send "heartbeats" periodically (configurable by TEM Administrator - default 5 minutes) to the TEM Relay of about 1 KB that contain the differences in the TEM Clients properties.
  • Downloads: Downloads occur only when a TEM Console operator sends out an action to tell a TEM Client to download and run a file. The frequency of this operation is completely dependent on the activity of the TEM Console Operators.

Note:

If the Tivoli Endpoint Manager infrastructure is properly configured, the traffic described is between the TEM Client and the TEM Relay (which is usually a high bandwidth connection) and the information between the TEM Relay and the TEM Server (sometimes a slow bandwidth connection) is minimal: gather/download information sent only once and posts are compressed and forwarded to the TEM Server.
Expected Traffic Volume:

  • Gather: Low - Perhaps 5 KB - 50 KB per day per TEM Client spread throughout the day.
  • Post: Low - Perhaps 10 KB - 50 KB per day per TEM Client spread throughout the day.
  • Downloads: High - If no actions are pushed, then no downloads will occur. If a patch is downloaded, the TEM Client will download the file (30 KB - 300 MB) from the TEM Relay.

Note:

This traffic varies heavily depending on usage. A very active deployment (TEM Console operators sending lots of actions) might see numbers that are 10x the numbers above.
Possible Worst Case Traffic:

In a properly configured Tivoli Endpoint Manager deployment, the TEM Relays always have a high-bandwidth connection to the TEM Relays to minimize network traffic on the slow network connections. The worst case scenario is if the TEM Relays are improperly configured or the if the TEM Clients or TEM Relays fail in some form while a large download is being pushed. For instance, if all the TEM Relays simultaneously failed while Windows XP SP2 (250 MB) action is sent, the TEM Clients will all begin to download the file from the TEM Server. In such a scenario, all the slow network connections between the TEM Clients and TEM Server will be flooded with HTTP traffic on port 52311.

This situation can be recovered from by stopping the action immediately in the TEM Console (depending on how busy the network is, this could take 2 mins - 2 hours), blocking TCP on port 52311 at the network routers, or by shutting down the TEM Server/TEM Relays (this can be done through TEM, thus eliminating the source of the download servers).
Network Controls: The Tivoli Endpoint Manager platform provides the following controls for this type of traffic:

  • Configurable bandwidth throttling to TEM Relay or TEM Clients
  • Configurable gather interval (default 1 per day per Fixlet site)
  • Configurable minimum time to wait between posts (default 15 sec)
  • Configurable temporal distribution (spread out downloads over time) per action
  • Ability to set "policy" to prevent computers from downloading files if they are not pointed at the proper TEM Relay
Other Notes: Traffic from server to relay is required for force refreshes and for Wake-On-LAN.

One of the major focuses of deploying the Tivoli Endpoint Manager infrastructure is to get the TEM Relays in locations that are near the TEM Clients so that WAN traffic is minimized and the vast majority of the network traffic is between the TEM Clients and the TEM Relays on a fast LAN connection. The Tivoli Endpoint Manager platform offers a number of reports to see the status of the TEM Clients and which TEM Relays they are pointing to and it is the job of the TEM Administrators to make sure the TEM Relay and TEM Clients are properly configured.
 
 

B

Network Traffic: UDP "new information" message
Purpose: When there is new information available such as new Fixlet messages or new actions, a UDP message is sent to the TEM Clients' last known IP addresses to tell them to gather the lastest data from the TEM Server.
Port: 52311 (configurable by the TEM Administrator at install time)
Protocol: UDP
Origination: The UDP messages are sent from the TEM Clients' immediate "parent", which can be either a TEM Relay or the TEM Server.
Expected Traffic Volume: Low - The UDP messages are sent from the TEM Relay to the TEM Client every time a new action is sent or a new Fixlet message is gathered. The TEM Relay will send 3 UDP messages to each TEM Client (to help ensure delivery). Each UDP message is about 60 bytes.
Expected Traffic Frequency: In an idle deployment (no actions being sent) there will be virtually no UDP traffic. In a very active deployment where many actions are sent, there will be 3 UDP messages sent per action per TEM Client (as a reference point, sending 100 actions over the course of a day is considered very high TEM usage). Note that if the TEM Relays "nearby" the TEM Clients, the UDP traffic will be sent on the LAN and the WAN will have very little UDP traffic.
Possible Worst Case Traffic: The worst case scenario for this type of traffic is for the TEM Relays to fail and send much more traffic than they are supposed. In such a scenario, the TEM Relays would be repeatedly sending UDP messages to each of the TEM Clients. There is a control built in for the TEM Relays never to send more than 500 UDP messages per second (configurable), but if there were many TEM Relays or many TEM Clients, this UDP traffic could potentially cause lots of network traffic or router usage. In such a situation, you would see massive amounts of UDP traffic on port 52311 from the TEM Relays.

To recover from this situation, UDP traffic on port 52311 could be blocked at the routers or you could shut off the TEM Relays (can be done through TEM).
Network Controls: TEM provides the following controls in the product for this type of traffic:

  • Configurable limit of the number of UDP messages sent at one time from a TEM Relay
  • Configurable limit of the amount of time to wait after sending UDP messages from a TEM Relay
Other Notes: It is not strictly necessary for TEM to use UDP traffic for the system to work properly, but the UDP messages allow the TEM Clients to be notified of new information as opposed to repeated polling by the TEM Clients. Thus, the UDP messages can help reduce overall network usage and increase response time.



UDP messages from the TEM Relay to the Wake-On-LAN forwarding computer are required for Wake-On-LAN to work properly.		  
 
 
 

C

Network Traffic: Tivoli Endpoint Manager Relay Selection
Purpose: In a well configured TEM network, each TEM Client is configured to communicate with a local TEM Relay; however, manually selecting the TEM Relay for each TEM Client can take a lot of time and is error prone. To solve this problem, the TEM Clients can be configured to find their closest TEM Relay based on the number of network hops. To find their closest TEM Relay, the TEM Client use the ICMP protocol (similiar to tracert).
Port: N/A (the ICMP protocol doesn't use a port)
Protocol: ICMP
Origination: Each TEM Client will send progressive "rounds" of ICMP packets to each TEM Relay with increasing TTLs until a TEM Relay responds. For example, in a network of 2 TEM Relays, one 1 hop away and one 2 hops away, the TEM Client will send a ICMP message to both with TTL 1 and will receive 2 "time exceeded" messages from the local router. The TEM Client will then send an ICMP message to both TEM Relays with TTL 2 and will receive one "time exceeded" message and one reply message. TEM Client will then choose the TEM Relay that is one hop away.
Expected Traffic Volume: Low - The volume of the ICMP traffic is directly porportional to the number of TEM Clients, number of TEM Relays, and number of hops to each TEM Relay. For example, in the example above with a TEM Client and 2 TEM Relays, the number of pings is 1 TEM Client * 2 TEM Relays * 2 hops to closest TEM Relay = 4 ICMP pings. Each ICMP packet and the response is about 50-60 bytes. Note that if TEM Relays are close to the TEM Clients, the total network traffic is reduced considerably than if the TEM Relays are far away from the TEM Clients.
Expected Traffic Frequency: By default, the TEM Clients attempt to find a better TEM Relay once every 6 hours (configurable) or every time the TEM Client is started.
Possible Worst Case Traffic:

The worst case scenario for this type of traffic is for a large number of TEM Clients to have a large number of TEM Relays that are very far away or for the TEM Client to fail in some way that makes them repeatedly send ICMP packets. In such a scenario, there would be high amounts of ICMP packets being sent very quickly from the TEM Clients to each of the TEM Relays.



To recover from this situation, you can block ICMP at the routers or turn off TEM Client autoselection (can be done through TEM in anywhere from 2 min to 2 hours depending on how busy the network is).
Network Controls: Tivoli Endpoint Manager provides the following controls in the product for this type of traffic:

  • Relay autoselection can be disabled
  • Configurable interval for when the TEM Clients perform autoselection
  • Configurable limit on the maximum number of ICMP packets to send out in a given time interval
  • Configurable limit on the maximum number of "rounds" to send out during relay autoselection
Other Notes: The TEM Client autoselection can be a huge savings of time and effort because it allows the network to change (new TEM Relays added, TEM Relays removed, routers go up or down, etc.) and the TEM Clients will automatically change with the newtork. However, if necessary, manual TEM Relay selection can be used too.
 
 

D

Network Traffic: Tivoli Endpoint Manager Console traffic
Purpose: To administer TEM, the TEM Console operators will connect to the database on the TEM Server computer.
Port:
  • 52311 for sending actions (configurable at install time)
  • Database connection port varies (usually 1433)
Protocol:
  • HTTP (TCP/IP) for sending actions
  • Database connection protocol varies (usually ODBC over TCP/IP)
Origination: The TEM Console will connect to the database on the TEM Server.
Expected Traffic Volume: High - There is a lot of information (computer properties, vulnerabilities, etc.) sent between the TEM Console and the database. The amount of traffic is usually measured in MB per minute, but can vary considerably depending on the size of the deployment. Note: there are usually very few TEM Consoles active and they are usually on a high-speed connection.
Expected Traffic Frequency: Whenever a TEM Console operator is administering TEM, the database will repeatedly be polled for new data.
Possible Worst Case Traffic: It would be unlikely that this type of traffic would cause a "catastrophic" situation because unlike the situations involving the TEM Clients, there are very few TEM Consoles. If a TEM Console user connected to the TEM Server and repeatedly hit "F5" to refresh the data, there would be a lot of traffic between the TEM Server and TEM Console.
Network Controls: Tivoli Endpoint Manager provides the following controls in the product for this type of traffic:

  • There is a "refresh rate" for each TEM Console user (default 15 seconds)
Other Notes: If the connection is slow between the TEM Console and the TEM Server, it is not easy to administer Tivoli Endpoint Manager so usually the TEM Console has a high-speed connection to the TEM Server.
 
 

E

Network Traffic: TEM Server gets new data from external IBM Fixlet servers and download files from various download servers (such as Microsoft)
Purpose: The TEM Server will periodically check to see if any new Fixlet messages are available from the IBM Fixlet server on the public Internet. Also, when a file needs to be downloaded, the TEM Server will download and cache the file from a download server (for instance, all Microsoft patches are downloaded from Microsoft's download servers)
Port: 80
Protocol: HTTP (TCP/IP) (sometimes FTP for downloads)
Origination: The TEM Server will connect to the IBM Fixlet servers or download servers on the public Internet.
Expected Traffic Volume: Low - The initial gather of all Fixlet sites will likely be less than 500 KB. Subsequent updates will usually be less than 1-2 KB of traffic. Downloads, however, can be quite large, but since the files are cached, they will be downloaded only one time.
Expected Traffic Frequency: The TEM Server will check for new Fixlet messages once every hour by default. Files will be downloaded whenever a TEM Console operator sends an action that requires a non-cached patch.
Possible Worst Case Traffic: It would be unlikely that this type of traffic would cause a "catastrophic" situation because unlike the situations involving the TEM Clients, there is only one TEM Server. If the TEM Server were to be sent many many actions or somehow malfunctioned, it might download a lot of files from the Internet, but it seems unlikely that this would cause a problem.
Network Controls: Tivoli Endpoint Managerprovides the following controls in the product for this type of traffic:

There is a configurable interval that the TEM Server will check for new Fixlet messages.
Other Notes: The TEM Server is the only component in a default Tivoli Endpoint Manager installation that contacts the Internet directly.


More information about TEM Relays can be found here and more general documentation about TEM can be found here.